Deep Reinforcement Learning for Practical Phase-Shift Optimization in RIS-Aided MISO URLLC Systems

We study the joint active/passive beamforming and channel blocklength (CBL) allocation in a non-ideal reconfigurable intelligent surface (RIS)-aided ultra-reliable low-latency communication (URLLC) system. The considered scenario is finite (FBL) regime problem solved by leveraging deep reinforcement learning (DRL) algorithm named twin-delayed deterministic policy gradient (TD3). First, assuming an industrial automation system, signal-to-interference-plus-noise ratio achievable rate FBL are identified for each actuator. Next, CBL optimization formulated where objective to maximize total all actuators, subject non-linear amplitude response at RIS elements, BS transmit power budget available CBL. Since highly non-convex non-linear, we resort employing actor-critic DRL based on TD3. method relies interacting with environment taking actions which phase shifts variables, expected observed reward, i.e., rate. assess performance loss of system when non-ideal, response, compare it ideal without impairments. numerical results show that optimizing shifts, beamforming, variables via TD3 outperforms conventional methods beneficial improving network considering size.